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Trellis Based Maximum Likelihood Signal Estimation Method and Apparatus 
for Blind Joint Channel Estimation and Signal Detection 

Field of the Invention 

The invention relates generally to communication systems, and more particularly 
to estimating wireless communication channels subject to multi-access 
5 interference, multi-path fading, varying power levels and noise in order to detect 
signals transmitted through the channels. 

Background of the Invention 

10 There are a number of problems with communication systems. A major challenge 
in signal recovery in mobile communications is to recover symbols from a signal 
transmitted over multiple channels in the presence of channel fading and distortion. 
In many communication systems, receivers observe the sum of multiple 
transmitted signals due to multi-paths, plus any noise. In addition, as a mobile 

15 transmitter proceeds along its route, the communication environment is constantly 
changing. That results in received signals that are displaced with respect to time, 
space, and frequency. Therefore, many wireless communication systems operate 
under highly dynamic conditions due to the mobility of the transceivers, varying 
environmental conditions, and the random nature of channel access. Equalization 

20 of multipath channels and detecting signals in receivers poses many difficulties. 

In wireless communication systems, mobile transmitters send symbols at a high 
data rate. Multiple copies of the signal, with delays, can interfere with the main 
signal. This is referred to as "delay spread," and causes inter-symbol interference 
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(ISI). As a result, equalizers may be required. Equalization with training sequences 
is widely used in many communication systems. However, in applications such as 
multipoint communication networks, it is desired to have synchronization and 
equalization without using a training sequence. 

5 

In fast fading channels, training sequences can be a large overhead and may 
significantly reduce channel throughput. Therefore, blind equalization techniques 
with joint channel estimation and signal detection is often required. Various 
methods for blind equalization are known in the prior art, including methods using 
10 stochastic gradient algorithm and higher order signal statistics. However, a primary 
drawback of those methods is the slow convergence of channel estimation, often 
requiring 10000-50000 symbols before an acceptable channel estimate is obtained. 

Maximum Likelihood Sequence Estimation 

15 

A widely used blind equalization method is based on maximum likelihood 
sequence estimation (MLSE). For a simplified discrete-time finite channel model 
h(k), k = 0, 1, L, given the transmitted symbols x(ri), n = l,2, N, the received 
sequence y(n), n- 1,2, N can be expressed as: 



where the variable v(n) is an independent and identically distributed (i.i.d.) additive 
Gaussian noise with zero mean and variance a 2 . For a block of Af received 
symbols, the probability density function of the received sequence y(n), 
conditioned on knowing the channel model h(n) and transmitted symbols x{n) is: 
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y{n) = ]T h(k)x(n-k)+v(n), 
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The blind equalization essentially estimates the channel impulse response h and the 
transmitted data* only from a set of observations y. Equivalently, the solution 
minimizes the following cost function over h and x, 

J(h,x)=f 4 \y n -th k x n _ k \ 2 =\\y-Ah\\\ (3) 



n-\ 



k=0 



where A is 



0 
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(4) 



10 When the input signal is known by minimizing equation (3), the maximum 
likelihood channel estimation can be obtained by: 

*ml« = (A'A^AV- (5) 

On the other hand, when the channel impulse response is known, the maximum 
15 likelihood sequence of the source symbols can be obtained using the Viterbi 

algorithm. When neither h and x are known, a direct approach is to determine the 
maximum likelihood estimate of the channel impulse response h for each possible 
data sequence x. Then, the data sequence that minimizes the cost function J(h,x) 
for each corresponding channel estimate is selected. Obviously, this exhaustive 
20 search approach has a large computational complexity that makes it impossible to 
use in practice when the number of symbols received increases. 
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The typical maximum likelihood based joint channel estimation and signal 
detection method generally includes initial channel estimation, followed by 
alternative signal detection and channel update steps. 

5 Typical prior art methods operate alternately between parameter estimation, 

assuming that the symbols are known, and a Viterbi sequence detection, assuming 
that the channel parameters are known. One potential drawback of those methods 
is that they are very sensitive to the initial guess of the channel parameters, e.g., 
see Kaleh et al., "Joint parameter estimation and symbol detection for linear or 

10 nonlinear unknown channels," IEEE Trans, on Communications, Vol. 42, No. 7, 
PP. 2406-2413, July, 1994, and Feder et al., "Algorithms for joint channel 
estimation and data recovery - application to equalization in underwater 
communications," IEEE Journal of Oceanic Engineering, Vol. 16, No. 1, PP. 42- 
55, January, 1991. 

15 

Quantized channel estimation methods can also be used. Those methods are 
generally less sensitive to the initial channel estimate, e.g., see Zervas et al., "A 
quantized channel approach to blind equalization" Proc. IEEE ICC'92, Vol. 3, pp. 
1539-1543, 1992. While that method has its advantage in a parallel structure and 
20 robustness to the initial estimate of the channel parameters, its complexity 

increases exponentially with the order of the channel due to operating multiple 
Viterbi algorithms simultaneously. 

Another prior art method for joint blind channel estimation uses parallel adaptive 
25 general Viterbi algorithm to determine multiple estimated channels for symbol 
detection, see Seshadri et al., "List Viterbi decoding algorithms with applications," 
IEEE Trans, on Communications, Vol. 42, No. 2/3/4, pp. 313-323, February, 1994, 
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and Seshadri "Joint data and channel estimation using blind trellis search 
techniques," IEEE Trans, on Communications, Vol. 42, No. 2/3/4, pp. 1000-1011, 
1994, and U.S. Patent No. 5,263,033 "Joint data and channel estimation using fast 
blind trellis search," issued to Seshardi on November 16, 1993. Because multiple 
5 channel estimates are maintained concurrently during each time instant, this 
method requires more computational resources than the conventional Viterbi 
algorithm. More important, that method does not guarantee the best convergence 
performance even though it can quickly achieve a rough channel estimate. That is 
because when channel estimation converges, noise may cause the method to select 
5 10 randomly from different channel estimates at each step. The channel estimates are 
'5 updated immediately after path metrics are determined. Therefore, a single high 
rE noise value can diverge the direction of channel update and result in a non-optimal 
f channel estimate. In addition, the number of trellis mappings derived by Seshadri 
s is 

U 15 N ue = 2 L (2 L+1 -2)(2 L+1 -4)...(4)(2)(1) = 2 2i (2 L )!/2. 

N| That value is much higher than the minimal number of trellis mappings desired for 
effective channel estimation. 

To summarize, the less complicated methods of the prior art channel estimation 
20 and symbol detection are sensitive to the initial channel estimate, while robust 
initial estimates are very computationally complex. 
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Summary of the Invention 

The invention provides a two-mode trellis based maximum likelihood (ML) 
method and apparatus for blind joint channel estimation and signal detection. 

5 During a first mode, multiple channels are estimated and symbols detected through 
the estimated multiple channels are used to update the estimates. As the symbols 
are detected, path metrics for each of the multiple channels and channel variations 
are monitored. While the minimum path metric of the multiple channels and the 
corresponding channel variations do not satisfy predetermined constraints, the 

10 process repeats. Otherwise, a best channel of the multiple channels is identified. 

When the best channel is identified, the method switches to a second mode where 
symbols are detected on the best channel as long as the minimum path metric of 
the best channel, or the rate of change of this minimum path metric remains below 
15 a predetermined threshold. If the minimum path metric of the best channel exceeds 
the threshold, the process switches back to the first mode of operation to find a 
new channel estimate. 

More particularly, the invention provides method and system for performing blind 
20 joint channel estimation and signal detection in a communication system that 
includes multiple channels for transmitting symbols. 

The multiple channels are estimated using a small number of symbols initially 
received through the multiple channels. 

25 
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When the channels have been estimated, a single best channel is selected. From 
then on, the bulk of the symbols are detected only on the best channel, while the 
best channel is updated. 

5 Brief Description of the Drawings 

Figure 1 is a block diagram of a blind channel to be estimated by the present 
invention; 

? 10 Figure 2a is a top-level block diagram of a method and apparatus for blind channel 
estimation and signal detection according to the invention; and 

M Figure 2b is a detailed block diagram of the method and apparatus for blind 
r channel estimation and signal detection as shown in Figure 2a; 

jl 15 

yl Figure 3 is a representation of a trellis structure according to the invention. 

Detailed Description of the Preferred Embodiment 

20 Blind Channel Estimation and Signal Recovery 

Figure 1 is a model of the "blind" channel problem solved by the invention. A 
source signal s 101 is transmitted through a channel H 1 10 subject to the above 
conditions. This results in an unknown signal y 102 having time and frequency 
25 dispersion. Additive noise n 120 further complicates the problem, leading to a 
received signal r 103. In classic channel estimation, both the input and output 
signals are usually known. However, in blind channel estimation, only the received 
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signal r 103 is available, and therefore, the effects of the channel H 1 10 and noise 
n 120 can only be blindly estimated. 

System Structure 

5 

Figure 2a shows a system 200 for blind joint channel estimating and signal 
detecting according to the invention. The system 200 includes a channel estimator 
and selector 210 and a symbol processor 240. The input to the system 200 is a 
received signal 201 and the outputs are a sequence of detected symbols 209 and the 
S 10 estimated channel response 25 1 . A switch 235 connects the input signal 201 either 
% to the channel estimator and selector 2 1 0 during a first mode of operation, or the 
:f; symbol processor 240 during a second mode of operation. The system 200 can also 
£ include a buffer 265 for storing delayed input for the symbol processor 240 while 
* the channel estimator and selector 210 is operating. 

M; 15 

Sf Figure 2b shows details of the channel estimator and selector 210 including a first 
U symbol detector 215 connected to a trellis-based multiple-channel estimator 220, as 
well as a status monitor 225 that receives output from the first symbol detector and 
the multiple a channel estimator. The status monitor 225 provides output to a best 
20 channel selector 230. 

The symbol processor 240 includes a second symbol detector 245 connected to a 
trellis-based best channel updater 250 with a delay 270, a minimum path metric 
determiner 255, and a multiple channel selector 260. 
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System Operation 

The system 200 according to the invention has two modes of operation. During the 
first mode, the channel estimator and selector 210 determines a best channel 231 
5 based on the minimum path metrics and the channel variations measured in a 
Viterbi trellis with updating multiple estimated channels for the input signal 201. 

During the second mode, the symbol processor 240 detects symbols received 
through the best channel and determines when to switch back to the first mode 
10 based on the minimum path metric of a Viterbi trellis using maximum likelihood 
techniques applied to the best channel using signal 261. 

Operation of the method 200 for blind joint channel estimating and signal detecting 
begins in the first mode where the switch 235 directs the input signal 201 to the 
15 channel estimator and selector 210. The first symbol detector 215 receives the 
input signal and estimates symbols x 216 traveling through multiple channels. An 
initial "rough" estimate of the channel impulse is used to detect the first estimated 
symbol. 

20 Thereafter, the multiple channel estimator 220 uses the output symbols x 216 of 
first symbol detector 215 to further estimate the impulse response H 221 of the 
multiple channels. The status monitor 225 uses the output from the first symbol 
detector and the multiple channel estimator to monitor a path metric of each of the 
multiple estimated channels and channel variations. A best channel selector 230 

25 selects the estimated channel 231 among the multiple estimated channels that has 
the minimum path metric and a channel evolution which satisfies a linear 
constraint, described in greater detail below. 



9 



MH-5081 
Cao et al. 



When the best channel 231 is selected, the switch 235 directs the input signal 201 
to the symbol processor 240. The second symbol detector 245 only detects symbols 
209 traveling through the best channel. The second symbol processor 245 output is 
5 a sequence of detected symbols jc 209. Further, a best channel updater 250 uses the 
delayed output 270 of the second symbol detector to further update the impulse 
response h 207 of best channel. The best channel updater output 207 is used in the 
second symbol detector to detect symbols 201. 

10 A path metric determiner 255 uses the output of either the second symbol detector 
or the best channel estimator to determine a minimum path metric of the best 
channel. If the path metric of the best channel is larger than a predetermined 
threshold, then the multiple channel selector 260 causes the switch 235 to direct 
the input signal 201 to the channel estimator and selector 210 with signal 261 so 

15 that a next best channel can be selected. 



During the first mode of operation, we use parallel list Viterbi algorithm to 
concurrently maintain multiple channel estimates. Then, we select the channel with 
the minimum path metric and correct evolution as the best channel. Either a least 

20 mean square (LMS) or a recursive least square (RLS) can be used to update 
channel estimates after symbols are detected. Only a small number of received 
symbols are required for this task, for example, less than fifty for channel lengths 
up to five. The linear constraint of trellis mapping is exploited to monitor the 
evolution of the channel estimates and the path metrics to determine when the best 

25 channel has been estimated. When the best channel is selected, the second 
operational mode uses adaptive Viterbi algorithm on the best channel only to 
update or track the best channel estimate and detect the symbols. 
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First Multi-Channel Mode of Operation 

Figure 3 models the trellis used by the multiple channel estimator 220 according to 
5 the invention. For a binary signal (M =2) and a channel impulse response L+l, a 
trellis structure 300 is defined. Figure 3 is an example for L=2. The task is to 
correctly map the received symbols to appropriate trellis branches. 

The number of the mappings is expressed by: 
1 10 N trellis = 2 L (2 L+1 -2)(2 L+1 -4)...(2 L+1 -2L), (6) 

1 where the number of terms in the above product is L+l . In the absence of additive 
It noise, the received channel symbols are 

% {C], C2, .--,C2, -C2 L ,..-, -C2, -C]}. 

^ 15 Equation (6) is interpreted as follows. The symbol Cj can be mapped to one of 2 L 
% branches when differential encoding is adopted. Then, the symbol -q is fixed 
i automatically due to the symmetry of the trellis branch. Therefore, symbol c 2 can 
be mapped to one of the following 2 L+1 -2 branches. Similarly, symbol c 3 is mapped 
to one of 2 L+1 -4 branches, etc. However, as long as symbol c L+1 is mapped, we are 
20 able to select L+l branches from the totally allocated 2(L+1) branches to set up 
L+l linear equations. From these linear equations, the L+l components of the 
channel impulse response are determined completely. Consequently, the rest of 
symbols {c L+2 ,..., c 2 L } and {-c L+2 ,..., -c 2 L } are automatically mapped to their 
corresponding branches according to the channel impulse response h. This property 
25 is the linear constraint, mention above, for mapping symbols to trellis branches 
according to the invention. 
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Notice that the linear constraint according to the invention has not been considered 
in the prior art. Also, the number of trellis mappings is much lower in our 
invention than in the prior art. While it is impossible to consider all of the trellis 
mappings to find the best one, we concurrently consider multiple "most- 
5 appropriate" mappings. This can be realized by incorporating the parallel list 
Viterbi algorithm with an LMS or RLS adaptive method. Limited by the linear 
constraint, the actual length of symbol sequence to get a good channel estimate is 
quite short. By employing parallel adaptive list Viterbi algorithm, fast convergence 
generally occurs in the first 30-50 symbols, when L < 4. 

Id 10 

S Second Single Channel Mode of Operation 

jj After the best channel is determined, it is unnecessary to continue using the 
? parallel adaptive list Viterbi algorithm. At this point, it is reasonable to switch to 

H> 15 the second mode of operation. While in this mode, either LMS or RLS processes 
M can be used to further update the best channel. 

In our method 200, the adaptive Viterbi algorithm is used to accomplish this task. 
A delay, equal to a small number of symbols, e.g., ten, is put between the symbol 
20 detection and further channel update. The delay reduces isolated high value noise 
that can diverge the updating of the best channel. 

We can use a buffer 265 to store input symbols while the best channel is estimated 
during the first mode. These stored symbols can then be detected during the second 
25 mode of operation. 
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In the second mode of operation, the invention performs symbol detection and 
channel update in only one pass of the trellis. This eliminates the long delay 
associated with the iterative prior art methods. The invention has better MSE 
convergence compared to continuous use of parallel adaptive list Viterbi algorithm 
5 with zero delay in updating channels, as in the prior art. The MSE of channel 
estimation can be expressed as: 

MSE(/) = \t£[hj - h(k) f , (7) 

/ ;=1 k=0 

where t is a time stage along the trellis processing. The variable / is the number of 
data blocks tested to obtain the average MSE value, the value h(k) is the original 
10 channel response, and h es j(k) (t) is the estimated channel response at time instant t 
for the i th block. 

Switch Between Above Two Operation Modes 

15 The linear constraint for the mapping of channel symbols to trellis branches not 
only guarantees a best channel selection after a small number of received symbols, 
but also provides the technique to switch between these two operation modes as 
well. 

20 As shown in Figure 3, in the absence of channel noise, we have eight different 
channel symbols {{c h -c 7 ), (c 2 , -c 2 ), (c 3 , -c 3 ), {c 4 , -c 4 )}, given a starting state, when 
four different symbols from above four different sets are received. The trellis 
mapping is completely fixed by receiving of the next symbol. Therefore, the length 
of the received sequence that is sufficient to determine the channel response can be 

25 as short as five symbols, in this example. These same characteristics can be 
adopted to monitor whether a best channel is obtained, or not. 
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We can check the path going through the states of the trellis with minimum path 
metrics at different stages. If the linear constraint is satisfied in a number of 
consecutive stages, then a high confidence of determining the best channel is 
5 ensured. This linear constraint also indicates whether one channel is in correct 
evolution. That is, whether the channel is updated from a previous version of its 
own, or from a previous version of another channel. In short, channel variations 
and the minimum path metrics can be exploited to determine when to switch from 
the first mode of operation to the second mode of operation. 

10 

In the second mode, the minimum path metric of the best channel update along the 
trellis is monitored. The value of the minimum path metric always increases, but 
the rate of increase and the absolute value of the minimum path metric are 
determined by the distance between detected symbols and original transmitted 
15 symbols. Therefore, if the rate of increase or value of the minimum path metric is 
larger than a predetermined threshold, then the current estimated channel response 
is diverging from the true channel response. In this case, the multi-channel selector 
260 asserts a signal on line 261 to switch back to the first mode of operation and 
select another best channel. 

20 

Effect of the Invention 

The invention of trellis based maximum likelihood signal estimation and blind 
joint channel estimation and signal detection has the following effects. First, the 
25 method is stable to initial channel estimation because multiple channels are 
concurrently estimated during the first mode. Second, our method has reduced 
computational complexity during the second mode of operation, when only a 
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single channel estimate is maintained in the trellis progress. Third, the bulk of the 
symbols are processed in only a single pass, except for a very small number of 
symbols used to estimate the best channel in the first mode of operation. This 
eliminates the long delay that is inherent in many prior art methods. Finally, the 
5 invention can achieve a high convergence performance for channel estimation. 

Although the invention has been described by way of examples of preferred 
embodiments, it is to be understood that various other adaptations and 
modifications may be made within the spirit and scope of the invention. Therefore, 
10 it is the object of the appended claims to cover all such variations and 
modifications as come within the true spirit and scope of the invention. 
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